It might fit in with Matthew 7:6 (“Do not give what is holy to dogs nor throw your pearls before swine.”)
Rather than ask what Jesus meant, ask yourself who were the writers and why did they say this? Whom were they talking about and talking to?
Always remember that not one of the NT writers had actually seen or heard Jesus. They were writing a generation later, and each writing group was operating with religio-political agendas that developed since Jesus' death.
We can safely say that the "Matthew" authors were part of the Jewish-Christian community. That group, originally based at Jerusalem, operated in opposition to Paul, who was based at Antioch. I do not know, but it is not impossible that the "Matthew" writing group were throwing a few barbs at their Gentile-Christian "brethren".
Looking at the Bible from the outside in provides a valuable understanding.
Doug